Skip to content

fix(common-utils): preserve boolean grouping for mixed AND/OR searches - #3690

Open
Tyagiquamar wants to merge 1 commit into
ClickHouse:mainfrom
Tyagiquamar:agent/fix-mixed-operator-precedence
Open

Tyagiquamar wants to merge 1 commit into
ClickHouse:mainfrom
Tyagiquamar:agent/fix-mixed-operator-precedence

Conversation

@Tyagiquamar

Copy link
Copy Markdown
Contributor

Fixes #3032

Summary

A search that mixes AND and OR without explicit parentheses now emits SQL that preserves the parse tree. a:1 AND b:2 OR c:3 previously generated ((a = '1') AND (b = '2') OR (c = '3')), which ClickHouse evaluates as (a AND b) OR c. It now generates ((a = '1') AND ((b = '2') OR (c = '3'))), matching the parser's a AND (b OR c) grouping.

Continues #3032 after the baseline coverage in #3131, as requested by the maintainer.

Root Cause

serialize() in packages/common-utils/src/queryParser.ts added parentheses only when a node was marked parenthesized in the source. The lucene parser right-nests mixed-operator chains, so a:1 AND b:2 OR c:3 produces an AND node whose right child is an unparenthesized OR node. The serializer flattened that child. SQL binds AND tighter than OR, so ClickHouse regrouped the predicate and returned rows that should have been excluded, silently.

Every log, trace, and dashboard search that mixed operators without writing parentheses was affected.

Solution

In the binary-AST branch of serialize(), an operand is now wrapped in parentheses when it is a binary node with a different operator and no parentheses of its own. Same-operator chains, nodes carrying a start (NOT) marker, and negating parent operators keep their existing spelling, so saved searches that do not mix operators are unchanged. No APIs or fields were added.

Added a patch changeset for @hyperdx/common-utils.

Regression test

Added CustomSchemaSQLSerializerV2 - mixed operator precedence to packages/common-utils/src/__tests__/queryParser.test.ts:

  • an unparenthesized mixed chain keeps the inner OR grouped;
  • unparenthesized and explicitly parenthesized forms emit identical SQL;
  • a same-operator AND chain is not over-parenthesized.

On unmodified main, the first two tests fail with ((a = '1') AND (b = '2') OR (c = '3')); they pass with this change.

How to reproduce

Build a SearchQueryBuilder for a:"1" AND b:"2" OR c:"3" with String columns a, b, c (as in the new test). On main the inner OR is ungrouped. On this branch it is grouped.

Test plan

Validation on native Windows, Node v24.22.1 / Yarn 4.13.0 (CI uses the .nvmrc Node 22.23.4; full-suite confirmation is left to GitHub Actions):

  • npx prettier --check on the two changed TypeScript files and the changeset: passed.
  • npx eslint src/queryParser.ts src/__tests__/queryParser.test.ts --max-warnings 80 (repo threshold): 0 errors, 18 pre-existing warnings in untouched lines: passed.
  • yarn workspace @hyperdx/common-utils tsc --noEmit: passed.
  • yarn workspace @hyperdx/common-utils jest src/__tests__/queryParser.test.ts --ci --coverage=false --runInBand --silent --verbose=false: 406/406 passed.

Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
@Tyagiquamar
Tyagiquamar requested a review from a team as a code owner November 17, 2026 10:24
@vercel

vercel Bot commented Nov 17, 2026

Copy link
Copy Markdown

@Tyagiquamar is attempting to deploy a commit to the HyperDX Team on Vercel.

A member of the Team first needs to authorize it.

@changeset-bot

changeset-bot Bot commented Nov 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 120b693

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@hyperdx/common-utils Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

// AND before OR, regrouping `a AND b OR c` as `(a AND b) OR c`. Only wrap a
// child whose operator differs; same-operator, start-marked and negated
// forms keep their current spelling so saved searches are unaffected.
const negatingOperator =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 major — Mixed chains with negated operators are still regrouped: a AND NOT b OR c keeps the bug this PR fixes

The lucene grammar parses AND NOT/OR NOT/NOT as single operators and right-nests them the same way it does AND/OR. So a AND NOT b OR c parses as AND NOT(a, OR(b, c)). Because of the !negatingOperator exclusion, the serializer still emits a AND NOT b OR c, which ClickHouse evaluates as (a AND NOT b) OR c. That is wrong: it should be a AND NOT (b OR c). Results for this query are still silently wrong, and it now behaves differently from a AND b OR c, which the PR does group. Fix: drop the negatingOperator exclusion so the OR operand is wrapped (NOT (b OR c)), or narrow it to a case the AST actually needs. Add tests for a AND NOT b OR c, a NOT b OR c and a OR NOT b AND c alongside the new mixed operator precedence cases. The current tests only cover AND/OR, so they would pass with this regression in place.

Nothing blocks merge automatically, but a maintainer will expect this fixed if it is a real defect in code this PR changes. If it is about surrounding code, reply and say so instead of patching. Do not widen the PR. How to respond

@github-actions

Copy link
Copy Markdown
Contributor

PR Review

If you are a coding agent acting for the author, read this first.
Nothing below blocks merge automatically; a maintainer decides. What they will expect:

  • 🔴 critical or 🟠 major in code this PR adds or changes: fix before asking for review.
  • 🔵 minor in code this PR changes: your call. Fix if small, otherwise reply.
  • Anything about surrounding code, or asking you to widen the change (hoist a helper,
    dedupe with another file, fix other call sites): never fix here, whatever the
    severity. Reply with one sentence; if it is critical, say so plainly so a human sees it.

One commit per review round. After two rounds, stop and ask a maintainer to review
scope rather than addressing more automated findings. Full rule: AGENTS.md.

1 finding(s): 🔴 0 critical · 🟠 1 major · 🔵 0 minor

1 posted as inline comment(s) on the changed lines.


Severity is the reviewer's own estimate and is used for ordering, not filtering. No finding blocks merge automatically; a maintainer decides.

@github-actions

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found.

This is a tightly-scoped correctness fix to the lucene→SQL serializer in packages/common-utils/src/queryParser.ts: a binary operand is now wrapped in parentheses when its operator differs from its parent's, so a mixed a:1 AND b:2 OR c:3 chain emits ((a='1') AND ((b='2') OR (c='3'))) instead of letting SQL re-group it as (a AND b) OR c. It ships a regression test for the primary case and a changeset. No P0/P1 defects were confirmed in the diff.

🟡 P2 — recommended

  • packages/common-utils/src/queryParser.ts:2203 — The two new wrap-suppression guards — a negatingOperator parent (NOT/AND NOT/OR NOT) and a hasStart child — are exercised by zero tests, so whether a negated mixed chain such as a:"1" AND NOT b:"2" OR c:"3" still regroups incorrectly in the emitted SQL is unverified; an incorrect grouping here silently returns the wrong rows.
    • Fix: Add serializer tests for a negated-parent mixed chain and a NOT-prefixed child in a mixed chain, each asserting the exact expected SQL string.
🔵 P3 nitpicks (4)
  • packages/common-utils/src/queryParser.ts:2322 — serialize() is shared with genEnglishExplanation, so the new paren-wrapping also changes English explanation output for mixed-operator queries, but no English test covers that path.
    • Fix: Add a genEnglishExplanation assertion for a mixed query like a:"1" AND b:"2" OR c:"3" to lock the output.
  • packages/common-utils/src/__tests__/queryParser.test.ts:2777 — The reverse ordering (an OR parent with an AND right-child, e.g. a:"1" OR b:"2" AND c:"3") and deeper 4+ term mixed chains are untested, leaving the differing-operator branch covered in only one direction.
    • Fix: Add the mirror case and a 4-term mixed chain asserting exact SQL.
  • packages/common-utils/src/queryParser.ts:2203 — negatingOperator is derived from the parent node's operator but sits beside per-child checks in serializeOperand, so its name reads as if it describes the child operand.
    • Fix: Rename to parentIsNegation (or operatorIsNegation) to make the guard's scope explicit.
  • packages/common-utils/src/queryParser.ts:2198 — The explanatory comment lists three keep-current-spelling cases but the condition has a fourth skip clause, !child.parenthesized.
    • Fix: Mention the already-parenthesized case in the comment so it matches the code.

Reviewers (6): testing, maintainability, project-standards, agent-native, learnings, previous-comments.

Testing gaps:

  • The correctness, adversarial, and kieran-typescript reviewers did not return results before synthesis; correctness conclusions here rest on static analysis of the diff plus the testing reviewer, and the lucene parser could not be executed in this environment to confirm the negated-mixed-chain grouping. Re-run those three (or add the P2 test above) before relying on this as a full correctness sign-off.
  • No coverage asserts previously-correct same-operator OR-only chains are byte-identical after the change; only an AND-only chain is pinned.
  • No prior learnings exist under docs/solutions/ for query-predicate correctness — this fix is a good candidate to capture once merged.

No finding blocks merge automatically. A maintainer will expect P0/P1 findings in code this PR changes to be fixed; P2/P3 are your call -- fix or reply. Never fix findings about surrounding code here; reply instead. Do not widen the PR. How to respond

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Search mixing AND and OR without parentheses returns wrong rows

1 participant